1. Field of the Invention
The present invention relates to an image processing apparatus for converting moving image data into moving image data having a higher frame rate, and a method of controlling the same.
2. Description of the Related Art
Conventionally, a CRT has been synonymous with a moving image display device for, for example, television. However, liquid crystal displays, plasma displays, and FED displays have been put into practical use in recent years. That is, there are various types of displays now.
Different types of displays adopt different display methods. For example, display devices based on a liquid crystal device (e.g., direct-view-type liquid crystal display device, liquid crystal rear projector, and liquid crystal front projector) can use many scanning methods. In any case, the light output period in each pixel portion occupies a large part of the display period of one frame. For this reason, such a display device is called a hold-type display device.
On the other hand, in, for example, a CRT or FED, light is output in each pixel portion once in a frame. The light emission time is much shorter than the frame display period and is normally 2 msec or less. For this reason, such a display device is called an impulse-type display device.
There also exist plasma displays and field sequential displays which are of types different from the above-described classifications.
The display methods of the respective types have the following features.
(1) Hold-Type Display Device
A display device of this type emits light during a large part of a frame period. Hence, the temporal imbalance of light intensity is small, and flicker is rarely observed. Additionally, pursuit (pursuing a moving portion in a moving image by eyes) makes moving blurring relatively large in accordance with the length of the light emission period in a frame. “Moving blurring” here is different from that caused by the response characteristic of a display device.
(2) Impulse-Type Display Device
A display device of this type emits light in a very short time during a frame period. Hence, the temporal imbalance of light intensity is large, and flicker synchronous with the frame rate is observed. However, movement blurring in pursuit is rarely observed. It is therefore possible to obtain a resolution almost equal to that of a still portion.
In general, the light emission period of a display device changes depending on the display method and display device. The above-described types (1) and (2) are quite different in terms of the light emission period. The longer the light emission period (corresponding to the hold time) is in each method, the larger the movement blurring in pursuit. The shorter the light emission period is, the smaller the movement blurring. That is, the light emission period and the magnitude of moving blurring are almost proportional to each other. On the other hand, concerning flicker synchronous with a frame, the longer the light emission period is, the smaller the flicker observed. The shorter the light emission period is, the larger the observed flicker. That is, the light emission period and flicker have trade-off relationships.
A solution to the two problems is multiplying the frame frequency by N. In many case, N=2. That is, the rate is doubled. When the frame frequency is doubled, the light emission period in each double-rate frame is halved. This also almost halves the movement blurring. Regarding flicker, if an initial frame frequency of 60 Hz is doubled to 120 Hz, the frequency of flicker falls outside the response characteristic of human eyes. Hence, no flicker is observed.
As described above, doubling the frame frequency (more broadly speaking, multiplying the frame frequency by N, where N is greater than 1) has a large effect, but poses a new problem.
For example, when the frame frequency of an original image signal is 60 Hz, the image information is updated every 1/60 sec. If the frame frequency is doubled to display image data at 120 Hz, necessary image information is missing every other frame. As a solution, identical images are displayed, for example, twice if the frame frequency is doubled. This eliminates flicker but cannot improve movement blurring in the original image. In an impulse-type display device, double images are observed by pursuit (this phenomenon will be referred to as “double-blurring”).
Two methods are mainly used to double the frame frequency.
The first method detects the motion of an object in an original image and estimates images between two frames. This is generally called an intermediate image generation method by motion compensation. This method is disclosed in, for example, Japanese Patent Laid-Open Nos. 2004-159294 and 2004-297719.
In the second method, first, a filter process is performed for each frame of an input image to separate a spatial high-frequency component associated with movement blurring and a spatial low-frequency component associated with flicker. The spatial high-frequency component is concentrated to one sub-frame (one of the two double-rate frames corresponding to the original frame). The spatial low-frequency component is distributed to both sub-frames (both of the two double-rate frames corresponding to the original frame).
In this specification, this method will be called a “method of separating an image into spatial frequencies and distributing them to sub-frames for display”.
The “method of separating an image into spatial frequencies and distributing them to sub-frames for display” is discussed in Japanese Patent Laid-Open Nos. 6-70288 and 2002-351382, and U.S. Pre-Granted Publication No. 2006/0227249A1.
However, the first and second methods have the following problems.
The first method has two problems. As one problem, an error may occur in vector calculation as a result of motion detection, and no means for correcting it is present (to be referred to as problem 1-1). The other problem is that the calculation amount becomes enormous in accordance with the image size (to be referred to as problem 1-2).
The second method also has two problems. As the first problem, since an image displayed by a first sub-frame and that displayed by a second sub-frame do not correctly reflect the display time difference between them, a pursuit image is distorted (problem 2-1). The second problem is caused by moving a component (actually, spatial high-frequency component) of one sub-frame to the other sub-frame. More specifically, this more easily saturates the sub-frame, and the effective dynamic range consequently becomes narrower than the proper dynamic range of the display device (problem 2-2).